Gap+uring in-Porrra+ion on Ar+s Pnr+ioipnn+9 
Exploring Engagement Fund Toolkit 



The Exploring Engagement Fund provides risk capital for arts nonprofits to experiment 
with innovative ideas about how to engage diverse Californians. In order to understand 
the variety of Californians engaged in arts experiences, this guide is intended to support 
current and future Fund grantees in collecting participant information. 
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Overview o-P Key S+ep<5 


Exploring Engagement Fund grantees are testing their approaches to engaging participants from communities 
of color and low-income groups — two populations historically underserved by arts nonprofits and of particular 
interest to The James Irvine Foundation. 

To assist Exploring Engagement Fund grantees with their data collection efforts, the Irvine Foundation worked 
with independent consulting firm Harder+Company Community Research to provide technical assistance to 
organizations and their projects. As a result of this work, Harder+Company has identified the following key steps 
as essential elements of information-gathering efforts. These steps are explained in depth on the following pages, 
and form a framework for planning and implementation. 


PLAN 



Identify Learning and Reporting Needs 
Determine How to Collect the Information 


Create User-Friendly Tools 


IMPLEMENT 



Collect the Information 


USE 



Analyze and Apply the Information 



Current grantees: Contact your Harder+Company technical assistance provider with any questions 
and requests for support— such as drafting specific survey questions or setting up an Excel template. 
Technical assistance providers also have resources on effective strategies for capturing participant 
information on race/ethnicity and income. We are here as a resource for your Exploring Engagement 
Fund information-gathering efforts. 

You'll see this icon at points in this document where we want to remind you that we are here for you! 
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iden+i-Py Learning 
and Repor+ing Need<? 


With your project team, discuss your project's reporting requirements, what you would like to learn, 
and how information can inform project and organizational decision making. 

For example, the Irvine Foundation asks Exploring Engagement Fund grantees to estimate participants' race/ 
ethnicity, as well as the percent of total participants who are part of a low-income community. Some grantees 
are also interested in assessing how their project impacts the people who participate. It is important to 
consider how different kinds of information can benefit your project and organization. 


Considerations 


Identify the learning interests and requirements of project 
partners and artists, your board of trustees and funders. 


I 


Prioritize what you want to learn. What is most needed, timely 
or required? What information will help inform your future 
work? Plan on collecting only the information that you will use. 


Vs/hn+ Do you Vs/nn+ \o Know^ 







Participant 
characteristics, such 
as race/ethnicity, age, 
gender or income? 


Required for Exploring 
Engagement Fund 
grant reporting 



New skills developed 
by participants, such 
as singing, drawing 
or telling their story? 


Participant satisfaction How participants 

and feedback on the heard about the event? 

event, and/or their 

likelihood of attending 

another event? 


Participant overall 
opinions and 
experiences? 


These examples are not a comprehensive list, but may be helpful as you think about the types of information that are important to your organization. 
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De+ermine How \o 
Colle<s+ +he In-Porma-Hon 



Your project team can consider a variety of information-gathering methods including surveys, focus 
groups and observations. Each method works well for collecting particular types of information, and 
each has time, equipment and financial implications. You can select one or more methods to collect 
participant information. 

Before selecting your information-gathering method(s), you will need to determine whether you can 
collect information directly from participants and/or if you will need other indirect alternatives. 


Asking someone direct questions about their age, race/ethnicity, income and other personal 
information is typically more successful when you: 1) have a relationship with the participant (e.g., 
they already attended multiple workshops or know your organization), and 2) offer an incentive to 
those who answer questions (e.g., a raffle or take-home goodies). Asking direct questions also provides 
an opportunity to capture participant contact information for future outreach efforts. Some situations 
are not ideal for direct questioning — for example, when participants are new to your organization 
or when you don't offer incentives. If you have concerns about alienating people or harming project 
participation by asking people direct questions, you should explore indirect information gathering. 

Indirect methods include gathering information from community partners, the U.S. Census Bureau, 
or other available reports. For example, if you have an event that mainly attracts local residents to a 
neighborhood park you can look at the Census data for that neighborhood to make informed estimates 
about race/ethnicity and income for your participants. 


No+e 

in-Porma+ion 
gathering 
should never 
hur+ or 
in+er-Pere 
wi+h The ar+<5 
experience 


Example Methods 


Survey 

A survey is a set of questions that are answered in 
a written, interview or online format. Surveys are 
good for learning information about participant 
characteristics (e.g., race/ethnicity or income) and 
opinions (e.g., satisfaction or engagement) from a 
large number of people. They can be conducted in 
person, by mail or online. 

Focus Group or Interview 

These techniques are guided by a set of questions — 
interviews are one-on-one, and focus groups engage 
multiple people. They are best used for obtaining 
feedback from a smaller group or learning how a 
program impacted some of its participants. 


<P> 


Observation 

In structured observations, data is collected 
by watching people at an event and recording 
information (e.g., by counting or assigning categories). 
Observations work best when you want to document 
behavior or responses that can be easily seen. 
Observations are not recommended for use in 
identifying participant characteristics such as race/ 
ethnicity and income. 

Graffiti Wall 

This technique invites people to write or draw 
responses to a posted question on a physical or 
virtual "wall." Graffiti walls are best used for collecting 
information about people's experiences or feedback. 
They allow people to respond using one word, multiple 
sentences or visual representations like drawings. 
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Crea+e U<5er- 
Friendly Toote 

Now that you have decided what you want to learn and report, and how you will collect that information, 
you are ready to design effective information-gathering tools. 

Each method for collecting information has different considerations. For example, surveys are more 
effective when they are relatively short, easy to read and in accessible language. On the other hand, focus 
group questions are usually only seen by the person conducting the session; however, questions should 
still be clear and easy to understand when asked to participants. Effective tools should be user friendly for 
your staff, relatively short and understandable by all participants. Several components (outlined below) 
will help you design effective information-gathering tools. 




Four Things \o Consider 


Questions 

Before writing questions for participants, consider 
their style, flow and accessibility. For each question, 
think about the kind of answers you expect to receive 
and how you will analyze the information. Thinking 
about analysis during this development phase will help 
ensure that you meet your learning priorities and can 
help you refine your questions and response options. 
Also, consider if you want to gather the information 
anonymously (no name attributed to responses). 
Anonymous responses may help people be more 
candid. However, identified responses can help you 
be responsive to individual feedback. See "Tips for 
Developing Good Questions" on the following page. 

Location 

Think about where you will be collecting information. 

Is it outside in a noisy area, or inside a quiet place 
where talking is discouraged? Consider how the 
location could impact how much time someone would 
be willing to spend answering your questions. Also, 
think about logistical barriers, such as the number of 
entrances and exits for the event and having a space 
for people to stand or sit to complete surveys. 


Testing 

Testing or piloting the questions with a few current or 
past participants is recommended. It can help highlight 
areas that need to be clarified or shortened. Listen 
carefully and embrace feedback. If it is not possible to 
test your questions with participants, you could have 
staff members who are not involved in the project 
review them. 

Sampling 

What is sampling? At events with a large number of 
people, information can be collected from a smaller 
group of people instead of all attendees. For example, 
at an event with 1,000 participants, you could conduct 
surveys with 100 to 200 attendees. Find a way to 
keep the sampling as random as possible in order 
to obtain feedback that represents the entire range 
of participants attending the event. For example, 
sampling could be conducted by surveying every fifth 
person who walks through the entrance to the event. 
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Tip<5 Por Developing 6rood Question*? 

Use these tips while planning and writing the questions you will ask project participants. Taking care 
with the following elements in advance can help you be more effective when gathering information. 


I. Be^in with warm-up pue< 5 +ion <5 \o make people -Peel coom-Portable. 


2. Put question*? in a lo^ioal order with related topic.*? near 
eaoh other. 


3. U«?e word*? and <?entenoe<? that are simple and ea*?y 
to underhand. 


4-. Consider the language^?) that participant*? *?peak, a*? well a*? their 
literacy and education levels - a *?ixth ^rade reading level i<? a 
^ood benchmark -Por mo*?t survey and -Pocu*? ^roup pue*?tion<?. 


S'. Think about how much time people will be willing to *?pend 
an*?werin <0 your question*?, and limit the number and type o-P 
question*?. Seven to 10 ojue*?tion<? work*? well. 


G. Think about how people will -Peel bein^ a*?ked the*?e pue*?tion*?. 
Consider i-P there are way*? that the question*? could be 
mi*?interpreted. 
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Collect +he In-Porma-Hon 


You are now ready to begin gathering information from participants! 

The first step is to train your staff and/or volunteers and set some parameters on how to collect 
information from participants. Keep in mind the type of event you are having, and be flexible. 
Sometimes data collection does not go as planned — to be well prepared, have back-up plans. 



Considerations 


Training Staff & Volunteers 

Provide clear training to the staff/volunteers collecting the information. Discuss with them the information- 
gathering approach, learning and reporting priorities, possible participant questions, logistics and supply 
needs (e.g., pens and clipboards), as well as potential challenges. People usually feel most comfortable 
answering questions from someone similar to themselves who is friendly and genuine. Select your 
staff/volunteers carefully. 

Providing Instructions to Participants 

When announcing or asking people to answer questions, it is important to tell them why you want to learn this 
information and provide clear instructions for how they can participate in the process. Let them know why it 
is important to your project, your organization, your work in the community and your funders. Let participants 
know that their input will benefit the project and your organization. 

Giving Incentives to Participants 

Incentives help motivate people to share their information and opinions. Incentives can be universally appealing 
items, such as being entered into a raffle to win a $50 gift card. Incentives may also be something unique to 
your project, such as a behind-the-scenes arts experience or branded merchandise. 
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Analyze and Apply 
+he in-Porma+ion 



After you have gathered the information from participants, it is time to make sense of your "data." 

Depending on the information-gathering techniques you used, the information will be in the form of survey 
responses, observations conducted by staff or volunteers, or notes from a focus group. The type of information 
you gathered will impact how you analyze it and what decisions it can help you make. Before you begin 
analyzing, determine your response rate. 

The response rate is the number of people who participated in the information gathering compared to the 
total number of people who participated in the entire event/activity. For example, if 100 people participated 
in your event and 20 people completed a survey, the response rate is 20 percent. It is important to highlight 
the response rate when sharing information with your funders, board or others, and when making decisions. 
Your organization might use the information from only five percent of participants differently than information 
drawn from a higher percentage of participants. You should strive for at least a 20 percent response rate. 


Analyzing Surveys, Foo u$ 6rroup<5 
and Interviews- Example Processes 


Surveys 

Create an Excel document with a column for each survey question, and responses from only one survey on 
each row. Responses should be entered consistently from survey to survey, and you should note if someone 
did not respond to a question or if they gave two answers when you only asked for one. Use Excel to calculate 
percentages of responses for each question across all surveys. Questions that required people to write in their 
answers can also be listed in Excel and analyzed in a manner similar to focus group/interview information below. 



Focus Groups & Interviews 

Review the notes from each focus group and list themes (common comments or opinions) from the discussion. 
After you note all the themes, review the notes again to count the number of times each theme appears. 

This will help you identify the stronger (most repeated) and weaker (less repeated) themes. 


Using your information: Organizations can use the information gathered in a variety of ways. While one 
primary use may be to report to your funders, we recommend that after the information is analyzed you 
review and discuss it as a project team. Some questions to consider and discuss include: 

• What is surprising about the information gathered? What is not surprising? 

• What information should inform our current and future programming? 

• What information should be shared with our board of trustees and funders? 

• What information is not clear and requires that more information be gathered? 

• What questions may have not been clear to participants? 

• How can we improve information-gathering efforts going forward? 
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harder+company 

community research 1 y 

Harder+Company Community Research is a comprehensive 
social research and planning firm with offices in San 
Francisco, Davis, San Diego, and Los Angeles, California. 
Harder+Company's mission is to help our clients achieve social 
impact through quality research, strategy, and organizational 
development services. Since 1986, we have assisted nonprofits, 
foundations, and government agencies throughout California 
and the country in using good information to make good 
decisions for their future. Our success rests on providing 
services that contribute to positive social impact in the lives 
of vulnerable people and communities. 

www.harderco.com 



The James Irvine 
Foundation 


The James Irvine Foundation is a private, nonprofit grantmaking 
foundation dedicated to expanding opportunity for the people 
of California to participate in a vibrant, successful and inclusive 
society. The Foundation's grantmaking focuses on three 
program areas: Arts, California Democracy and Youth. Since 
1937, the Foundation has provided over $1.3 billion in grants to 
more than 3,500 nonprofit organizations throughout California. 
With about $1.9 billion in assets, the Foundation made grants of 
$73 million in 2014 for the people of California. 

www.irvine.org 
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